The invention has particular relevance to an electronic imaging system in which the input images are either stationary images or time-varying sequences of images, and in which the output images are subject to a static or dynamic digital zoom. The static zoom effects size changes in a stationary image, and the dynamic zoom effects continuing size changes, often on a frame-to-frame basis in a video environment, on a time-varying sequence of images. The zooming process, which may be implemented in a spatial interpolator, such as described in copending Ser. No. 817,204, "Digital Image Interpolation System for Zoom and Pan Effects" by Keith R. Hailey (filed Jan. 6, 1992 in the name of the same assignee as the present invention), may be required for spatial standards conversion, or for image magnification within a given spatial standard.
The effect of zooming upon sampled image data must be carefully considered in order to avoid the problem of aliasing. As known to those skilled in this art, an image can be characterized by a two-dimensional parameter known as spatial frequency. According to the Nyquist sampling theorem, in order not to lose information contained in a signal, the signal must be sampled at a sampling frequency (f.sub.s) that is equal to at least twice the bandwidth of the signal. The actual bandwidth is ordinarily established by prefiltering the analog signal before it is sampled, so that the bared edge of the signal does not exceed the upper frequency limit, which is called the Nyquist frequency, required for sampling.
In further considering the effect of zooming, it should be intuitive that, if the image is compressed, the lines appear to the viewer to become closer together, and the spatial frequency of the image is thus increased. This has the ancillary effect of increasing the upper band edge of the image's spatial frequencies. If the upper band edge now exceeds the Nyquist frequency, information is lost and aliasing results. This is handled in the prior art by inserting, before compression, a variable bandwidth digital filter that is responsive to a scaling factor representative of the degree of compression to accordingly band-limit the digital signal, and thereby to avoid any aliasing that might result from subsequent compression (see, for example, U.S. Pat. Nos. 4,760,605; 4,805,129; and 4,660,081). If the image is enlarged, so that lines appear to the viewer to become further apart, the upper spatial frequency of the image is decreased and the upper band limit is accordingly decreased. According to the prior art (e.g., the aforementioned U.S. Pat. No. 4,660,081) this is not a problem, and the variable bandwidth filters are simply disabled to provide an essentially straight through signal path.
Unlike straightforward band limiting filters, which provide a low pass function, the provision of image, or sharpness, enhancement requires digital filters having a maximum magnitude at a spatial frequency related to the output Nyquist frequency of the system. Such filters are typically one-dimensional or two-dimensional Finite Impulse Response (FIR) bandpass filters that are programmed in a variety of conventional ways to multiply a sequence of input image samples by a set of coefficients, and to provide the sum thereof as the output of the filter. The coefficients, which provide a bandpass characteristic, are generated in a conventional manner, such as an optimized frequency sampling design technique found in Theory and Application of Digital Processing, by L. R. Rabiner and B. Gold, Prentice-Hall: 1975, pp. 105-123. Another useful technique for generating a generalized bandpass filter is described in "Optical flow using spatio-temporal filters", by D. J. Heeger, International Journal of Computer Vision, 1988, pp. 279-302. The latter filter is referred to as the Gabor filter.
Enhancement is most commonly performed on an image signal after any spatial interpolation, or zooming, is done that might affect the bandwidth of the system, and after the system is accordingly band-limited to prevent aliasing, or otherwise reconverted to the original band of frequencies present before zooming. This allows one set of filters to be designed for the various spatial frequencies that are to be enhanced. However, because of signal-to-noise considerations, it is generally most beneficial to do enhancement on luminance information; likewise it is most effective to do spatial interpolation on full band color signals, rather than separately on the luminance and color/chroma information in the signal. Consequently, in applications where separate luminance and color signals are generated, it is desirable to do enhancement on the luminance signal before the luminance and color signals are processed into full band signals for spatial interpolation, that is, before zooming. Since the process of zooming affects the pass-band of the image signal spectrum, and consequently the spectrum must be band-limited to prevent aliasing, a situation arises wherein a fixed set of enhancement filters will affect different spatial frequencies as the degree of zoom is changed. For example, a bandpass enhancement filter designed to pass high band spatial frequencies without zoom will affect frequencies above the band edge after the image spectrum is compressed and band-limited. This will produce the effect of gradually diminishing enhancement as the degree of zoom is reduced, until the "enhanced spectrum" is entirely outside the band, and thus unnoticeable.
While the maximum magnitude of an enhancement filter may be correctly positioned within the Nyquist pass-band by a number of effective methods (described above), the problem remains that the enhancement filters may not be properly positioned at band edge after zooming. The aforementioned Gabor filter suggests the usefulness of a tunable peak frequency. Applying such teaching to the problem of zooming leads to several additional problems. The Gabor filter can be viewed as a sine wave multiplied by a Gaussian window in the spatial domain. The width of the filter is determined by the window, and the peak frequency by the sine waves. In order to define the peak frequency of a Gabor filter, it is necessary for the Gaussian window to envelope a significant number of sine wave cycles at the required frequency. This has two implications which become progressively more serious as the peak amplitude decreases in spatial frequency. First, the number of taps in the filter becomes unrealistic to implement in hardware. Secondly, because the width of Gaussian windows in the spatial and frequency domains are inversely proportional to each other, a Gabor filter designed to peak at low frequencies will, by definition, have a very narrow pass-band; this may not be appropriate for the application. As a result, the problem of frequency positioning of enhancement filters prior to zooming has not been effectively dealt with.